1. Field of the Invention
The present invention relates to a waveform data time expanding and compressing device.
2. Discussion of Background
Devices having a sound source (including devices with only a sound source), such as electronic musical instruments, store PCM waveform data with respect to several different musical intervals in order to prevent the storage capacity of a memory from becoming huge for memory of the waveform data of live musical instruments. When such devices reproduce a musical interval different from the stored musical intervals, the devices read out PCM waveform data by a pitch different from the original pitch for reproduction.
When PCM waveform data are read out by a pitch different from the original pitch, not only the pitch but also the sounding time changes. For example, it is assumed that a musical tone, which has a rise-time of 0.1 sec and an entire sounding time of 1 sec, exists at C4. When the musical tone is read out an octave lower by the pitch of C3, the rise-time is 0.2 sec and the entire sounding time is 2 sec. Conversely, when the musical tone is read out an octave higher by the pitch of C5, the rise-time is 0.05 sec and the entire sounding time is 0.5 sec.
The waveform of usual musical tones is composed of an initial rising part (a heading part or an attacking part), followed by a steady part (a sustaining part). Most musical tones pose features in the transition at their rising parts, which means that, when the time of the rising part significantly changes from the original one, an unnatural impression is given to a listener. In other words, when the rise-time significantly changes in a case wherein waveform data are read out by a pitch different from the original one as stated earlier, the reproduced musical tone is heard as an unnatural sound.
In order to solve this problem, it has been carried out that waveform data are divided into plural frames having a certain length (a fixed length whatever the waveform data are), that a divided frame is repeated with joints cross-faded for time expansion, and that a divided frame is eliminated with joints cross-faded for time compression.
Since the length of the divided frames is arbitrarily determined in this process, it has been frequently experienced that the repetition of a frame is heard (produces a jarring noise), or that the cross-fading generates phase interference to degrade the tone. This process has created a problem that producing a sound in real time is not appropriate since the load to be processed is great and since it takes much time to carry out the process.
The present invention is provided in consideration of these problems. It is an object of the present invention to provide a waveform data time expanding and compressing device capable of reproducing a natural sound without degrading the tone and giving an unnatural impression to a listener, and reproducing a musical tone in real time under time expansion or time compression.
The present invention provides a waveform data time expanding and compressing device comprising a waveform memory for storing data of a PCM waveform; a block address memory for storing addresses of respective blocks, the respective blocks having a length equal to a wavelength of a pitch as a trend of the PCM waveform or an integral multiple thereof; a parameter determining unit for determining an expansion and compression parameter; and a waveform reproducer for carrying out waveform reproduction the waveform by determining a reading number for waveform data in a certain block in response to the expansion and compression parameter, sequentially reading out the block addresses according to the determined reading number, and reading out the PCM waveform data based on the block addresses thus read out.
By the arrangement according to the present invention, the waveform reproduction is carried out by not only sequentially reading out waveform data having a certain length as one unit (one block) but also repeatedly reading out or thinning out waveform data as required to expand and compress the entire sounding time. Specifically, when the waveform data are read out with the pitch raised, a certain unit of the waveform data is repeatedly read out with respect to the rising part and its vicinities to expand the sounding time to make the sounding time closer to that of the original waveform data since the entire sounding time otherwise becomes shorter. Conversely, when the waveform data are read out with the pitch lowered, a certain unit of the waveform data is thinned out with respect to the rising part and its vicinities to compress the sounding time to make the sounding time closer to that of the original waveform data since the entire sounding time otherwise becomes longer. Even if the waveform data are read out with the pitch changed, a listener can have a natural impression by, in particular, making the sounding time of the rising part closer to that of the original waveform data.
The reason why the blocks of the waveform data to read out are determined to have a length equal to a wavelength of a pitch as a trend of the PCM waveform or an integral multiple thereof is that the respective blocks are required to be coupled as a continuous waveform even at the junctions since the respective units (blocks) to read out are finally coupled to produce a waveform. The reason why the blocks of the waveform data to read out are determined based on the wavelength of the pitch as the trend of the PCM waveform is that the wavelength of a keynote or a harmonic tone that determines a tone is set to be equal to the wavelength of the pitch as the trend to prevent the tone from degrading. If a harmonic tone of a PCM waveform has a greater amplitude and plays a greater role in determination of the sound rather than the keynote of the PCM waveform, the pitch of the harmonic tone may be determined as the pitch as the trend.
In detail, the respective blocks are determined to have a length equal to the wavelength of the pitch as the trend or an integral multiple thereof, the addresses in the respective blocks are stored, and the waveform data are read out in the respective block addresses (a certain block is repeatedly read out, or a specific block is thinned out).
The wavelength of the pitch as the trend of the PCM waveform may be found by an autocorrelation function as an example. Specifically, one reason why it is preferable to find the pitch wavelength by the autocorrelation function is that when a waveform is shifted to establish correlation, it is easy to extract the pitch as the trend since the point that is shifted by the pitch wavelength as the trend has the highest correlation value due to interference. There is another reason that it is not necessary to consider a sampling frequency when the finding is carried out by the autocorrelation function. The finding by the autocorrelation function is effective when a keynote has a low level and is subjected to a vibrato as in, particularly, PCM waveforms.
It is preferable that the respective blocks to read out have a length between zero-cross points. The reason is that when the respective blocks are repeatedly read out or thinned out, the junctions become smoother, and an unnatural impression is eliminated as the junctions are closer to zero-cross points.
In the specification, the phase "zero-cross points" are ones (e.g., Point 11) that has an amplitude value of not greater than zero at the current sampling point and is followed by the next sampling point having a positive amplitude value (Point 12) as shown in FIG. 2 explained later on. The reason is to make phase matching easy by determining only zero-crossing from negative to positive as the zero-cross points.
In addition, the expansion and compression parameter is set to determine the reading number for waveform data accordingly. The reason is that when the waveform data are read out with the pitch raised, the entire sounding time becomes shorter, and conversely when the waveform data are read out with the pitch lowered, the entire sounding time becomes longer, and that in order to make the shortened or lengthened sounding time closer to the original one, when the waveform data are read out with the pitch raised, a certain unit of the waveform data is repeatedly read out with respect to the rising part and its vicinities to expand the sounding time, and conversely when the waveform data are read out with the pitch lowered, a certain unit of the waveform data is thinned out with respect to the rising part and its vicinities to compress the sounding time, which have been stated earlier. To accomplish this, the expansion and compression parameter is set to determine the reading number for waveform data accordingly, allowing at least one block to be repeatedly read out or thinned out of course, the block of the waveform data to be read out is determined based on the pitch wavelength (or an integral multiple thereof) as the trend of the PCM waveform as stated earlier. Even if the waveform data are read out with the pitch changed, a listener can have a natural impression by, in particular, making the sounding time of the rising part closer to that of the original waveform data.
The determination of the reading number for the waveform data based on the determined expansion and compression parameter may be carried out by making accumulation of the expansion and compression parameter in every reading of the addresses, setting the reading number at an integral part of an accumulation value given by the accumulation, setting the integral part at 0 after determination of the reading number, and repeating the accumulation, as will be explained with reference to the embodiment stated later. However, the determination of the reading number for the waveform data is not limited to this method. If this method is employed, the expansion and compression parameter is determined so that the parameter is 1 for reproducing the PCM waveform as it is, the parameter is greater than 1 for reproducing the PCM waveform by a higher frequency, and the parameter is less than 1 for reproducing the PCM waveform by a lower frequency.
When a PCM waveform is read out by a pitch greatly different from the original pitch, the formant is significantly shifted, and a listener may have an unnatural impression. It is known that the formant is not shifted even if a person speaks with the pitch changed. The present invention also provides an arrangement capable of preventing formant from shifting even when a PCM waveform is read out by a pitch different from that of the original waveform as stated earlier. Specifically, the present invention provides a waveform data time expanding and compressing device comprising a formant component memory for storing a formant component of data of an original PCM waveform; a waveform memory for storing the PCM waveform data with the formant component removed; a block address memory for storing addresses of respective blocks, the respective blocks having a length equal to a wavelength of a pitch as a trend of the PCM waveform or an integral multiple thereof; a parameter determining unit for determining an expansion and compression parameter; a waveform reproducer carrying out waveform for reproduction by determining a reading number for waveform data in a certain block in response to the expansion and compression parameter, sequentially reading out the block addresses according to the determined reading number, and reading out the PCM waveform data based on the block addresses thus read out; and a formant adder for adding a corresponding formant to the reproduced waveform.
After the formant component of the original PCM waveform data is stored and the process stated earlier is carried out, the corresponding formant is added to the reproduced waveform. As a result, even if the pitch changes, a listener can have a natural impression since the formant is prevented from shifting.
When musical intervals, which are assigned to keys in, e.g., an electronic keyboard instrument, are reproduced, the expansion and compression parameter may be modified as to match with the musical intervals of the respective keys. In detail, the expansion and compression parameter set at the parameter determining unit may be modified by a key scale parameter.
Even if this arrangement is applied, noise could cause in repeated sections or junctions in a waveform when the wavelength of the pitch as the trend of the waveform is not an integral word (when a sampling frequency is 44100 Hz and a pitch is 438.8 Hz, the wavelength has 100.5 words, which is not an integral word). This is because there is a jump in the amplitude value at the repeated sections or the junctions in the waveform.
In such a case, cross-fading may be carried out at the repeated sections on the junctions to make the entire portions of the reproduced waveform merge smoothly. Specifically, for reading out the waveform data at the waveform reproducer, sample data of certain points following a zero cross-point are inputted into a buffer after completion of reading out the waveform data in a block, and sample data of first certain points in a subsequent block and the sample data in the buffer are cross-faded in reading out the waveform in the latter block.
As explained, the waveform data time expanding and compressing device according to the present invention can provide superior advantages in that when PCM waveform data are read out by a pitch different from the original pitch, a natural sound can be reproduced without degrading its tone and giving an unnatural impression to a listener, and that the time expanding and compressing reproduction of a musical sound can be carried out even in real time.